AI023
Introduction to Triton Programming
Reductions, Softmax, and Numerical Stability
Learning Objectives
- Execute and interpret reduction operations across multi-dimensional tensors.
- Formulate the Softmax function to transform logits into probability distributions.
- Diagnose common numerical stability issues such as floating-point overflow and underflow.
- Implement the numerically stable Softmax using constant shifting and the Log-Sum-Exp trick.